PROFICIENT VS. PREPARED 2018: 
DISPARITIES BETWEEN STATE TESTS AND 
THE 2017 NATIONAL ASSESSMENT OF 
EDUCATIONAL PROGRESS (NAEP) 


Achieve 


In today's economy, preparation for career or college is essential. Yet too few students graduate with the knowledge or skills to 
directly enter college or a career. The time students spend in on-the-job training or catching up in remedial courses cost students 
and their families more both in time and money. Many of these students are caught off guard because until this point, they 
received signals that indicated readiness for their next steps. 


The signals that schools send to students and families should be accurate and honest representations of preparedness for 
postsecondary options. One important measure is the annual assessments administered to students in English language arts 
(ELA) and mathematics. Historically, states have set low performance standards on those annual assessments, inflating the 
number of students who meet grade-level standards and sending signals of readiness to students who may not have reached 
a level of college and career readiness. This trend has changed over the last few years as states adopted rigorous, college- and 
career-ready standards and developed high-quality, aligned assessments. 


Achieve released two earlier reports comparing proficiency rates on state tests and the National Assessment of Educational 
Progress (NAEP). NAEP has long been considered an independent measure of student achievement that can be compared across 
states. For the original report, we analyzed the proficiency gap between state assessments and NAEP to determine if states 

were reporting proficiency rates on their own state assessments, using their state-determined standards, that were significantly 
different from proficiency rates on NAEP. In our original report, looking at 2013 state assessment and NAEP proficiency rates, we 
noted gaps of more than 40 points in 14 states, signaling that many state tests mislead the public about whether students are 
proficient. More students were scoring proficient on a state test than on the NAEP. 


By the time of our second report's release, looking at 2015 state assessment and NAEP proficiency rates, most states had made 
the transition to new, more rigorous tests and set proficiency or cut scores that reduced or even eliminated the proficiency 

gap. Parents, students, and teachers in the states where gaps closed were getting information from state tests that were much 
closer to other proficiency indicators, allowing them to make more informed decisions for their individual students. Additionally, 
policymakers and the general public had better information with which to make decisions to improve school systems. States 
deserved a lot of credit for setting rigorous cut scores. We urge states to continue to hold the line on setting a rigorous bar for 
student proficiency. 


With the recent release of the 2017 NAEP scores, Achieve is releasing an updated report on the proficiency gap. Across states, 
the gap between 2015 and 2017 on average has remained roughly the same (+/- 3 points), though there are several concerning 
outliers where gaps are now considerably larger than in 2015. While some states have transitioned state assessments, most have 
retained performance standards on those assessments that have not significantly widened the gap between proficiency on state 
and NAEP assessments from our last report with 2015 proficiency rates. A majority of states have narrow proficiency gaps of 15 
points or fewer between state and NAEP proficiency in both 4th grade and 8th grade ELA and mathematics, with nearly all states 
having a narrow gap in at least one grade or content area. Smaller gaps indicate that states are more accurately reflecting student 
proficiency. 


FINDINGS: 


¢ Small to moderate proficiency gaps in most states: In 2017, while gaps increased slightly overall, a large majority of 
states had gaps of 15 points or fewer in at least one grade or content area. 


e 26 states plus DC had gaps of 15 points or fewer in both grades and content areas 


¢ 48 states plus DC had gaps of 15 points or fewer in a least one grade or content area 


¢ Large gaps remain in some states: Thirteen states have gaps larger than 20 points in at least one grade or content 
area. Three states — lowa, Texas, and Virginia — have gaps of 20 or more points in 4th and 8th grade in both ELA and 
mathematics. 


¢ Closing the gap: Seven states — Massachusetts, Missouri, Nebraska, Ohio, Oklahoma, Tennessee, and Wisconsin — 
narrowed the proficiency gap in at least one grade or content area by 10 or more points between 2015 and 2017. 


¢ Some backsliding: Three states — Alaska, Arkansas, and Ohio — widened the proficiency gap by 10 or more points in at 
least one grade or content area between 2015 and 2017. 


The charts on the following pages show Achieve's analysis of proficiency gaps on state assessments and NAEP. The first set of 
four charts note the disparity in proficiency rates between 2016-17 state assessments and 2017 NAEP. The second set of four 
charts display the 2015 and 2017 proficiency gaps between state assessment proficiency rates and NAEP proficiency rates, with 
red arrows indicating a widening in the gap and green arrows indicating a narrowing of the gap. 
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Disparity in data reported by states and the 
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Note: The results reported for Texas reflect the aggregate percentages of students Approaching Grade Level, Meeting Grade Level, and Mastering Grade 

Level standards that have historically been used for school accountability and public reporting on student proficiency. However, as reflected in the state's 

ESSA plan approved in May 2018, moving forward Texas will consider only Meeting Grade Level and above to be proficient. This will change will likely 3 
narrow the gap in the future between proficiency on the state assessment and NAEP. 


4TH GRADE READING, 2016-17 
Disparity in data reported by states and the 


National Assessment of Educational Progress (NAEP) 
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Note: The results reported for Texas reflect the aggregate percentages of students Approaching Grade Level, Meeting Grade Level, and Mastering Grade 
Level standards that have historically been used for school accountability and public reporting on student proficiency. However, as reflected in the state's 

A ESSA plan approved in May 2018, moving forward Texas will consider only Meeting Grade Level and above to be proficient. This will change will likely 
narrow the gap in the future between proficiency on the state assessment and NAEP. 


8TH GRADE MATH, 2016-17 
Disparity in data reported by states and the 
National Assessment of Educational Progress (NAEP) 
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Note: The results reported for Texas reflect the aggregate percentages of students Approaching Grade Level, Meeting Grade Level, and Mastering Grade 

Level standards that have historically been used for school accountability and public reporting on student proficiency. However, as reflected in the state’s 

ESSA plan approved in May 2018, moving forward Texas will consider only Meeting Grade Level and above to be proficient. This will change will likely 5 
narrow the gap in the future between proficiency on the state assessment and NAEP. 


8TH GRADE READING, 2016-17 
Disparity in data reported by states and the 


National Assessment of Educational Progress (NAEP) 
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Note: The results reported for Texas reflect the aggregate percentages of students Approaching Grade Level, Meeting Grade Level, and Mastering Grade 
Level standards that have historically been used for school accountability and public reporting on student proficiency. However, as reflected in the state's 

6 ESSA plan approved in May 2018, moving forward Texas will consider only Meeting Grade Level and above to be proficient. This will change will likely 
narrow the gap in the future between proficiency on the state assessment and NAEP. 


PROFICIENCY GAP — 2014-2015 VS. 2016-2017 GAPS IN STATE-NAEP 
PROFICIENCY RATES 


4th Grade Mathematics 


2015 Differential 2017 Differential CHANGE IN GAPS 
(State- NAEP) (State-NAEP) 2015-17 

Alabama -22 -21 v 
Alaska -4 -9 a 
Arizona -4 -13 a 
Arkansas 8 -22 a 
California -6 -9 a 
Colorado 13 8 - 
Connecticut -3 -10 a 
Delaware -10 -14 a 
District of Columbia 3 0 * 
Florida -17 -16 v 
Georgia -4 -9 a 
Hawaii -8 -10 a 
Idaho -5 -6 a 
Illinois 9 8 * 
Indiana “15 -14 v 
lowa -36 -31 v 
Kansas 5 2 * 
Kentucky -9 -8 by 
Louisiana ~3 -|2 a 
Maine 1 -3 a 
Maryland 9 5 * 
Massachusetts 7 4 7 
Michigan -7 -6 y 
Minnesota -17 -14 y 
Mississippi 3 -4 a 
Missouri -11 -14 a 
Montana -1 -3 a 
Nebraska -3] -28 y 
Nevada N/A -10 N/A* 
New Hampshire 2 -3 * 
New Jersey 7 3 * 
New Mexico 9 4 ® 
New York -8 -8 


“These states have closed the proficiency gap by having state proficiency rates equal to, or more rigorous than, NAEP proficiency rates in both 2015 and 2017. 


**Nevada experienced significant technical problems with the administration of its state assessment in 2015, and student results are not considered 
representative. Therefore, we have not included the 2015 results in this analysis. 


4th Grade Mathematics (continued) 


2015 Differential 2017 Differential CHANGE IN GAPS 
(State-NAEP) (State-NAEP) 2015-17 

North Carolina -12 -17 a 
North Dakota 2 2 * 
Ohio -20 -31 a 
Oklahoma -35 -6 a 
Oregon -] -10 a 
Pennsylvania 1 -3 a 
Rhode Island 10 4 * 
South Carolina -13 -14 A 
South Dakota -4 -7 a 
Tennessee -10 -5 v 
Texas -29 -33 i 
Utah -] -7 

Vermont -2 -5 a 
Virginia -37 -31 v 
Washington -] -12 a 
West Virginia -2 -8 a 
Wisconsin -3 3 

Wyoming -3 -] a 

4th Grade Reading 
2015 Differential 2017 Differential CHANGE IN GAPS 
(State-NAEP) (State-NAEP) 2015-17 

Alabama -9 -9 

Alaska -10 -11 a 
Arizona -12 -19 a 
Arkansas -2 -12 a 
California -1] -14 a 
Colorado -3 -4 a 
Connecticut -12 -11 v 
Delaware -17 -18 a 
District of Columbia 1 -4 a 
Florida -15 -15 

Georgia -3 -6 a 
Hawaii -19 -16 wb 
Idaho -10 -10 


8 *These states have closed the proficiency gap by having state proficiency rates equal to, or more rigorous than, NAEP proficiency rates in both 2015 and 2017. 


Ath Grade Reading (continued) 


2015 Differential 2017 Differential CHANGE IN GAPS 
(State-NAEP) (State-NAEP) 2015-17 

Illinois -2 -2 

Indiana -30 -25 ev 
lowa -39 -38 a 
Kansas -20 -16 7 
Kentucky -12 -12 

Louisiana -11 -17 i 
Maine -11 -16 a 
Maryland -3 -2 by 
Massachusetts -] 3 v 
Michigan -18 -12 v 
Minnesota -19 -18 y 
Mississippi -4 -4 

Missouri -23 -27 a 
Montana -4 -12 a 
Nebraska -4] -18 v 
Nevada N/A -15 N/A 
New Hampshire -10 -13 a 
New Jersey -8 7 v 
New Mexico -1 =| 

New York 3 5 a 
North Carolina -21 -19 v 
North Dakota -] -1 a 
Ohio -34 -24 v 
Oklahoma -37 -8 v 
Oregon -15 -15 

Pennsylvania -18 -21 a 
Rhode Island 2 -2 a 
South Carolina 0 -12 a 
South Dakota -9 -13 a 
Tennessee -12 -4 v 
Texas -43 -40 v 
Utah -2 2 v 
Vermont -6 -6 

Virginia -34 -36 a 
Washington -15 -16 a 
West Virginia -15 -15 

Wisconsin -13 -13 

Wyoming -19 -23 a 


*These states have closed the proficiency gap by having state proficiency rates equal to, or more rigorous than, NAEP proficiency rates in both 2015 and 2017. 9g 
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8th Grade Mathematics 


2015 Differential 2017 Differential CHANGE IN GAPS 
(State-NAEP) (State-NAEP) 2015-17 

Alabama -10 -14 a 
Alaska 6 5 - 
Arizona 1 -3 am 
Arkansas 8 -18 a 
California -6 -] a 
Colorado 18 / i 
Connecticut -1 -6 a 
Delaware 5 -10 a 
District of Columbia 2 5 * 
Florida -19 -17 v 
Georgia -9 -10 a 
Hawaii -9 -11 aS 
Idaho -3 -3 

Illinois 1 -1 a 
Indiana -14 -17 a 
lowa -40 -37 7 
Kansas 10 10 sd 
Kentucky -16 -20 a 
Louisiana -14 -8 be 
Maine 2 1 * 
Maryland 12 16 - 
Massachusetts -2 2 y 
Michigan -3 -2 v 
Minnesota -10 -12 a 
Mississippi -6 -15 an 
Missouri -10 0) a 
Montana 5 1 i 
Nebraska -30 -24 A 
Nevada N/A 10 N/A* 
New Hampshire 2 0 ‘s 
New Jersey 22 16 - 
New Mexico 12 7 * 
New York 9 13 : 
North Carolina -10 -11 a 
North Dakota 4 6 * 
Ohio -18 -15 v 
Oklahoma -30 2 he 


*These states have closed the proficiency gap by having state proficiency rates equal to, or more rigorous than, NAEP proficiency rates in both 2015 and 2017. 


8th Grade Mathematics (continued) 


2015 Differential 2017 Differential CHANGE IN GAPS 
(State-NAEP) (State-NAEP) 2015-17 
Oregon -9 -] y 
Pennsylvania 6 6 - 
Rhode Island 20 16 * 
South Carolina -6 -8 a 
South Dakota -4 -4 
Tennessee -25 -2 uy 
Texas -43 -5] a 
Utah -4 -4 
Vermont 2 -2 a 
Virginia -36 -34 v 
Washington -] -6 v 
West Virginia -4 -6 a 
Wisconsin 2 13 * 
Wyoming -12 -10 v 
8th Grade Reading 
2015 Differential 2017 Differential CHANGE IN GAPS 
(State-NAEP) (State-NAEP) 2015-17 
Alabama -18 -18 
Alaska 0) -12 a 
Arizona -4 -4 
Arkansas ee -20 “a 
California -17 -17 
Colorado -3 -2 y 
Connecticut -11 -10 y 
Delaware -18 -19 a 
District of Columbia -6 -10 a 
Florida -25 -20 by 
Georgia -8 -8 
Hawaii -21 “17 v 
Idaho -15 -13 v 
Illinois -3 -| v 
Indiana -26 -21 b 
lowa -41 -38 v 
Kansas 5 10 * 
Kentucky -18 -23 a 


*These states have closed the proficiency gap by having state proficiency rates equal to, or more rigorous than, NAEP proficiency rates in both 2015 and 2017. ii 
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8th Grade Reading (continued) 


2015 Differential 2017 Differential CHANGE IN GAPS 
(State-NAEP) (State-NAEP) 2015-17 

Louisiana -17 -19 a 
Maine -12 -13 a 
Maryland -3 -] ¥ 
Massachusetts -18 0 y 
Michigan -16 -14 v 
Minnesota -16 -20 a 
Mississippi -11 -9 v 
Missouri -22 -25 A 
Montana -8 -13 a 
Nebraska -39 -13 vy 
Nevada N/A -18 N/A 
New Hampshire -13 -13 

New Jersey -11 -12 a 
New Mexico -3 -4 a 
New York -2 -1] a 
North Carolina -23 -21 v 
North Dakota -9 -1 a 
Ohio -32 11 v 
Oklahoma -46 -6 wy 
Oregon -21 -19 v 
Pennsylvania -19 -19 

Rhode Island 0) 0 . 
South Carolina -19 -10 v 
South Dakota -13 -13 

Tennessee -17 0) y 
Texas -50 -56 a 
Utah 5 -3 v 
Vermont -10 -9 a 
Virginia -39 -39 

Washington -20 -17 v 
West Virginia -16 -17 a 
Wisconsin -14 4 v 
Wyoming -16 -16 


*These states have closed the proficiency gap by having state proficiency rates equal to, or more rigorous than, NAEP proficiency rates in both 2015 and 2017. 


